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ABSTRACT 


This publication provides the algorithmic definitions and performance 
characterizations for a high performance adaptive "coding module" currently under 
development in custom VLSI at two NASA centers. Operation of at least one of these 
(single chip) implementations is expected to exceed 500 Mbits/s under laboratory 
conditions. Operation of a companion "decoding module" should operate at up to half 
the coder's rate. The functionality provided by these modules should be applicable to 
most of NASA's science data. 

The module incorporates a powerful adaptive noiseless coder for "Standard 
Form" Data Sources (i.e., sources whose symbols can be represented by uncorrelated 
non-negative integers where the smaller integers are more likely than the larger ones). 
Performance close to data entropies can be expected over a "Dynamic Range" of from 
1.5 to 12-14 bits/sample (depending on the implementation). 

This is accomplished by adaptively choosing the best of many "Huffman 
Equivalent" codes to use on each block of 16 samples. Because of the extreme 
simplicity of these codes, no table lookups are actually required in an implementation, 
thus leading to the expected very high data rate capabilities already noted. The 
"coding module" can be used directly on data which has been "pre-processed" to 
exhibit the characteristics of a Standard Form Source. Alternatively, a built-in 
Predictive Pre-processor can be used where applicable. This built-in Pre-processor 
includes the familiar one-dimensional predictor followed by a function which maps the 
prediction error sequences into the desired standard form. Additionally, an External 
Prediction can be substituted if desired (e.g., two-dimensional applications), further 
extending the module's generality. 
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ALGORITHMS FOR A VERY HIGH SPEED 
UNIVERSAL NOISELESS CODING MODULE 


INTRODUCTION 

References 1-4 provide the development and analysis of some practical 
adaptive techniques for efficient noiseless (lossless) coding of a broad class of data 
sources. These have been applied, in various forms, to numerous applications. 

Those functions and algorithms most desirable for incorporation in a "coding 
module" which could be implemented using current custom VLSI capabilities were 
presented at the first NASA Data Compression Workshop at Snowbird, Utah, in 
198815], A workshop committee recommended that NASA should proceed and 
implement this "coding module." Since then, both the Jet Propulsion Laboratory (JPL) 
and the University of Idaho (in conjunction with the Goddard Space Flight Center) 
have implemented multiple custom VLSI versions of this coder in CMOS. Operation of 
at least one of these (single chip) coding modules is expected to exceed 500 Mbits/s 
under laboratory conditions. This far exceeds performance expectations anticipated in 
1988. A companion single chip "decoding module" developed by the University of 
Idaho is expected to run at up to half the maximum rate of the coding module. 

It is anticipated that the high performance functionality of these modules can 
serve most of NASA's science data where a lossless representation is appropriate. 

The intent of this publication is to provide a concise description of the 
algorithmic and performance characteristics that are embodied in the coding modules 
of these VLSI implementations. A more general development, including extensive 
application notes can be found in Ref. 6. Details on these implementations are 
provided in Refs. 7-9. 
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THE CODING MODULE 


A functional block diagram of a general purpose lossless coding "module" is 
shown in Fig. 1. A variation in Rice’s original notation (of subscripting the Greek letter 
y) name various coding operations. Subsequent sections will quickly converge to 
more specific definitions that relate to the VLSI modules being implemented. 


A PRIORI 


or 

SIDE INFORMATION 



EXTERNAL | 
PRE-PROCESSED 
DATA 


Fig. 1. General Purpose Noiseless Coding Module Block Diagram 


The input to this coding module 

X n = x-| X2 . . . xj (1) 

is a J sample block of n bit samples. Y is a priori or Side Information that might help in 
the coding process. 
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The overall unspecific process of representing X n is named PSI7+ so that the 
actual coded result is 


PSI? + [X n ,Y] 

As Fig. 1 shows, the coding process is split into two independent steps as 
discussed below. 

STEP 1 

A Reversible Pre-processor is a process designed to convert the source 
represented by X n sequences (and Y) into a close approximation to a STANDARD 
FORM Data Source, represented by 8 n sequences. This process usually includes a 
de-correlation procedure (prediction). 

The pre-processor converts each X n (and corresponding Y, if any) into 


5 n ’ = 5 1 8 2 ...8j (2) 

a J > 1 sample sequence of n' bit samples. Usually n = n\ and we will henceforth 
assume that here. 

Standard Form Source. Specifically, 

a) Samples of 8 n are the non-negative integers 0, 1 , 2 q (3) 

b) Samples of 8 n are independent (4) 

c) With pj = Pr[8j = i], the probabilities are ordered so that the smaller 
integers occur more frequently, i.e., 

PO > pi > P2 > • ■ • (5) 
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d) The source entropy is given as 


H 5 =-ypi log 2 Pi bits/sample (6) 

i 

The best pre-processor will meet these conditions and produce the lowest H 5 . 

STEP 2 

An adaptive entropy coder, named PSI? for now, efficiently represents 
(Standard Source) pre-processed 5 n sequences with the coded result 

PSI? [5 n ] = PSI? + [X, Y] 


This entropy coder is independent of the pre-processor. Its goal is to achieve 
performance that remains close to H 5 as it varies with time. 

Note that the coding module in Fig. 1 allows for coder PSI? to be used directly 
on externally supplied pre-processed data. 

Prelude to the Details 

A general form of an Adaptive Entropy coder (designed to efficiently represent 
Standard Data Sources), which chooses from multiple algorithm options on a block- 
by-block basis, will be identified in the next section. Specific sets of such code options 
will be defined and incorporated in this structure as a parametrically defined 
adaptive coder. In doing so, the unspecific "PSI?" will be turned into a specific coder 
called "PSIss." 

Finally, the specific parameters of PSIss that are used in current VLSI 
implementations will be identified. 

ADAPTIVE ENTROPY CODER FOR THE STANDARD SOURCE 

PSI? of Fig. 1 represents the general purpose adaptive coder called PSI11 in 
Refs. 2-4. Basically, such a coder chooses one of a set of Code Options (coding algo- 
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rithms) to use to represent an incoming block of "pre-processed" data samples. A 
unique binary identifier precedes a coded block to tell a decoder which decoding al- 
gorithm to use. The following discussion will identify specific code options. 

Code Options 

Backup. When no coding of any form is performed on the data, we call this 

PSIbu 


PSIbu [8 n ] = 8 n (7) 

This representation is used in an adaptive coder when all other available code options 
fail to compress 8 n . 

The Fundamental Sequence Code. Recall that the pre-processed 
samples of 8 n are the non-negative integers i > 0. A variable length "Fundamental 
Sequence Code, fs", is defined for each i as follows 

i zeroes 

fs[i] = 000 0 0 0 1 for i > 0 (8) 

That is, simply append a 1 to the end of a sequence of i zeroes. 

The "Fundamental Sequence" itself is the application of fs[.] to all the samples of 
8 n . Following Rice's notation, 1 

PS1 1 [8 "] = fs[8 1 ] * fs[8 2 ] * . . . . fs[8j] (9) 

is the Fundamental Sequence. This defines Code Option, PSI1. 


Mn asterisk, *, is used to emphasize the concatenation of sequences. 


5 


Split-Sample Modes. The code option definitions here are basically to "split" 
off the k least significant bits of each 5 n sample and send them separately. The 
remaining n-k most significant bit samples are then coded using PSI1. Specifically, 
with 

5 n = 8-| 82 ■ • • 5j 


Let 

M n > k = m-|m2 mj (10) 

be the sequence of all the n-k most significant bit samples of 5 n and let 

Lk = Xsb-| * isb2 * isbj (11) 

denote the corresponding sequence of all the k-bit least significant bit samples of 8 n . 
That is 

6j = mj * isbj (12) 

The "Split-Sample" Mode Code Option PSIl.k is defined by 

PSI1 ,k[5 n ] = PSI1 [M n > k ] * Lk (13) 

Note that k = 0 is a special case where 

PSI1.0 = PSI1 


and when k = n 


PSIln, = PSIbu 
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The Individual Code Words. Note that the individual code word 
assigned to 5j in (12) when code option PSI1,k is applied is given as 

fs[mj] * isbj (14) 

That is, the Fundamental Sequence Code, fs[ ] in (9), is applied to the most significant 
n-k bits of 5j, followed by the least significant k bits of 5j. From this description, it should 
be easier to see that the only variable length code operation ever required is 
the application of fs[-], since the "fsbs" can simply be shifted out, and fs[ ] can be 
implemented without any table lookups. 

Performance of the Individual PSI1,k Options. Under certain familiar 
assumptions on the type of data source (these assumptions will be described in a later 
section), 

the individual "variable length codes" represented by (14) 

can be shown to be equivalent to Huffman Codes.! 1 °J (15) 

Thus they are not only extremely simple, they are optimum too. 

But even more important for their application in an adaptive coder, the entropy 
where PSIl.k achieves its best performance is at 

_k . 

Hg = k + 2 bits/sample ( 1 6) 


and performance remains close to Hg over a range of ±0.5 bit/sample. Thus there is at 
least one PSI1,k option that should provide efficient coding for any 

H5 > 1 .5 bits/sample (17) 

Such conclusions can also be drawn directly from simulations using a broad 
range of data sources. 
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Split-Sample Adaptive Coder, PSIss 

We can now use these Split-Sample Modes described above to replace the 
general coder, PSI? in Fig. 1 with a specific class of parametrically defined adaptive 
coders, named PSIss. PSIss is based on the following parameters: 

J = block size > 1 

n = Input Bits/Sample 0®) 

N = No. of Code Options 

X £ 1 (Dynamic Range Parameter) 

A functional block diagram is shown in Fig. 2. 

The representation of J sample 8 n , using an N option PSIss with parameter X > 
1 is given by 

PSIss[8 n ] = ID(id) * PSI1 , k(id)[8 n ] (19) 


where 


id » 0, 1, 2, ... N-1 (20) 

is the integer value of a coder identifier for the options used, and ID(id) is its standard 
binary representation, requiring 2 


[log 2 N) bits 


( 21 ) 


2 \z] is the smallest integer, greater than or equal to z. 
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PSIss 



Fig. 2. Parametric Split-Sample Adaptive Coder Functional Block Diagram 
PSI1,k(id) is the Split-Sample option specified by id, where 

( n for id = N-l 

k(id) = | (22) 

/ X - 1 + id Otherwise 

for parameter X > 1 . 

Dynamic Range. Except for limiting cases, the range of entropies where 
PSIss can be expected to efficiently represent pre-processed 8 n sequences has been 
shown to be closely specified by 

( n 

^ + 0.5<H5<min s (23) 

l X + N - 0.5 
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A close look at this expression shows that each increase in X moves the 
Dynamic Range of efficient performance upwards by 1 bit/sample. 

Performance Graph. A graph of typical performance for PSIss with N = 12, X 
= 1 , n = 1 4 and J = 1 6 is shown in Fig. 3. 



Fig. 3. PSIss Average Performance for N = 1 2, n = 1 4, X = 1 , J = 1 6. 


Choosing the Right Option. The optimum criteria for selecting the best 
option to use to represent 8 n is to simply choose the one that produces the shortest 
coded sequence. That is, 


10 




choose k = k* if 3 


^(PSIl.k* [5"]) = "JJ" {5?(PSI1 ,k [5 n ])} (24) 


Now, letting 


Fk =^[PSI1 [M n . k ]) 


we have from (12) 


^(PSI1,k[5 n ]) = F k + Jk 


(25) 


and from (8) - (10) 


J 

F k = 5>i + J (26) 

i=1 

(i.e., the sum of the most significant n-k bit samples plus the block size). 

Thus (25) and (26) can simplify the decision making in (24) without actually 
coding the data. But this can be further simplified. 

By taking advantage of the randomness in the least significant bits, the 
expected value of F k can be related to Fo by[ 4 l>[6] 

e{ F k |Fo } =2-k F 0 +5 (l -2- k ) (27) 


which we use as a estimate in (25). We have 


3 J/?(Z) = length of sequence Z in bits. 
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Y 1jk (5n) = 2-kF 0 +^(l-2-k) + Jk 
»^(PSI1,k [5 n ]) 


( 28 ) 


We can then choose k = k* if 

Yt ,k* (8 n ) = l 'Ij n { Yl ,k (5 n ) } (29) 

and this leads to distinct decision regions based solely on Fo (which by (26) can be 
determined by adding up the original samples since mj = 5j). The boundaries to 
adjacent PSIl.k decision regions are given by 

Fo=^ + J (2 k+1 ) bits (30) 

Any PSI1,k option will generate more bits than PSIbu when 

Fo > f [ (n-k) 2 k+1 + 1 - 2 k ] bits (31 ) 

A sample table of decision regions is shown in Table 1 for an N = 8 option 

PSIss. 

Note that if the largest value of k used is k = i, then PSI1, i will generate more 
bits than PSIbu when 

? l >(n-i) Jbits (32) 

which may be a simpler test than (31) provides in some cases. 
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Table 1 . Decision Regions for an N = 8 Option PSIss 


Code 

Option 

Fq Region in bits 

BB 

F, 5 5J/2 

PSI1,1 

SJ/2 < F, <, 9J/2 

PSI1,2 

9J/2 < F, 1 17J/2 

PSI1.3 

lTJ/2< F, <, 33J/2 

PSI1,4 

33J/2< F, < 65J/2 

PSI1,5 

65J/2< F, <129J/2 

PSIl.B 

1 29 J/2 < F 0 < (128n-831) J/2 

PSIbu 

(128n-831)J/2< F, 


PSIss Implementation Parameters 

The PSIss parameters used in the current VLSI implementations are as 
follows^. 9]; 

a) J = Block size of 5 n = 15 or 16 

b) n = Maximum Quantization of 5 n samples 

= 12 for JPL chips 
= 14 for University of Idaho chips 
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c) N = number of code options 

= 11 for JPL chip 
= 12 for University of Idaho chip 

d) X = Dynamic Range Parameter = 1 = Starting Option Parameter 

THE PRE-PROCESSOR 


The entropy coder, PSIss, as described in the previous section, was designed 
to efficiently represent (pre-processed) Standard Data Sources. PSIss doesn't need to 
know which pre-processor was used to produce its input. However, for an extremely 
broad set of real problems, the general pre-processing function of Fig. 1 can be 
replaced by the more specific Basic Predictive Pre-processor in Fig. 4. It is shown 
imbedded within the complete coding module for your convenience. 


Standard Predictor 

The first part of this pre-processor is a very simple predictor consisting of a 
single sample delay element. With xj as the i th sample in X n , this delay element 
"predicts" that xj equals the previous sample: 

Xj = xj_i (33) 

The previous sample could be the last sample from a previous block when coding 
multiple blocks. It is assumed that the sample delay is always initialized with a 
prediction. But note also at this time that the module's design allows for an External 
Prediction to be supplied in place of this simple one-dimensional form. 
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CODING MODULE, PSIss+ 



- J 

External 

Prediction 


Fig. 4. Basic Predictive Pre-processor within Coding Module, PSIss+ 


The difference between any sample and its prediction produces the error signal 

Aj = xj - xj (34) 

and the block of J error values 

A = Ai Aj (35) 
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As a new data source, A sequences tend to be uncorrelated and display a unimodal 
distribution around zero (when data values are not near the boundaries of its dynamic 
range). That is 4 


Pr [Aj = 0] > Pr [Aj = -1]>Pr[Aj = +1]> Pr [Aj = -2] > (36) 


The Mapper (into Standard Form Source) 

When the latter condition in (36) is true, the following function will map each Aj 
into a corresponding Standard Source 8j such that 

P0 > pi > p2 ^ P3 ^ 


Additionally, it will assure that an n-bit/sample xj from X n produces an n-bit/sample 5j. 
Further, the desired probability ordering of the 8j is more closely approximated when xj 
values are near 0 or xmax = 2 n - 1 . 

The Mapper is defined by: 

! 2 Aj 0 < Aj < 0 

2 | A j 1 - 1 -0 < Aj < 0 ( 37 > 

0 + | A j | Otherwise 


4 An equally valid assumption is 


Pr [Aj = 0] > Pr [Aj = +1] > Pr [Aj = -1] > Pr [Aj = +2] > 


but we will stick with (36). It appears to offer a slight advantage in the implementation 
of the mapping function described following (36). 
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where 


0 = min (Xj , xmax -Xj) 


xmax = 2 n - 1 


(38) 


A Further Benefit. Suppose the pre-processor in Fig. 4 is set to receive an 
external prediction, and fix that prediction to 

Xj = 0 (39) 


Then, tracing through Eqs. (37) and (38), one finds that 

Si = xj (40) 

That is, the input to the pre-processor, X, is passed directly through unchanged as 5. 
This means that the separate desired external input line in Fig. 1 (to allow the pre- 
processor to be skipped) can be omitted. 

The Ideal Case 

If one assumes that the distribution of A samples in (36) fits the Laplacian form, 
then the code equivalence result in (14) can be proven. H 0] That is, 


the simple PSI1,k codes are equivalent 
to Huffman Codes for Laplacian 

distributions of prediction errors. (41) 


Reference Sample 

Most of those applications which make use of the built-in Predictive Pre- 
processor will occasionally need to incorporate a "Reference Sample" along with the 
coded prediction errors (e.g., at the start of an image line). Each of the VLSI 
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implementations! 7-9 ! incorporates an optional feature to extract such a Reference 
Sample from an incoming data stream. 

STANDARDIZATION OF THE PSlss+ MODULE 

References 8 and 9 describe separate VLSI implementations of the coding 
module PSIss+. Both efforts represent significant achievements. However, as a result 
of earlier less specific definitions, these implementations have subtle variations in their 
algorithmic specification. To ensure compatibility in future applications, recent NASA 
efforts have focused on standardizing the definition of a PSIss+ Coding Module. 
This publication represents the most up-to-date statement of that effort. 
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